Image coding method, image coding device, image processing apparatus, and image transmitting/receiving system

ABSTRACT

An image coding device encodes image data in units of encoding line number that is number of lines necessary for implementing encoding. The image coding device includes: an image-data transforming unit that transforms image data of lines, number of which is smaller than the encoding-line number, into to-be-encoded image data of the encoding-line number of lines; and an encoding unit that encodes the to-be-encoded image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to and incorporates by referencethe entire contents of Japanese Patent Application No. 2014-143420 filedin Japan on Jul. 11, 2014.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image coding method, an image codingdevice, an image processing apparatus, and an imagetransmitting/receiving system and, more particularly, to a process forencoding image data with consideration given to a delay in transmissionof encoded image data.

2. Description of the Related Art

It is common to transmit data representing a moving image or a stillimage (hereinafter, collectively referred to as “image”) captured usinga security camera, a vehicle-mounted camera, or the like to an imagereceiving apparatus over a network. The amount of data transmitted inthis manner increases in proportion to image quality such as resolution,frame rate, and bit depth. However, because the bandwidth of a networkfor use in such transmission image data is limited, a restriction isimposed on quality of image data to be transmitted.

To enable transmission of image data of higher quality with such alimited network bandwidth, it is general compress image data using acodec such as a JPEG (joint photographic experts group) codec or anH.264 codec and transmit the compressed image data.

It is proposed to packetize image data, which is transformed byline-based wavelet transformation and is it units of a precinct, byrearrangement for each frequency component for the purpose oftransmitting the image data with low delay and efficiently, (see, forexample, Japanese Laid-open Patent Application No. 2008-028511).

An image compression process is typically performed in units of a fixeddata amount. For example, a JPEG compression process is performed byaccumulating image data in buffers or the like until the image data ofat least 8 lines is stored, and performing the compression process inunits of accumulated image data of 8 lines. When the compression processis performed in units of a determined number of lines in this manner,the compression process waits for the image data of the necessary numberof lines to be buffered, which can cause a delay in transmission ofcompressed image data and impair real-time transmission. A similarproblem arises as well in the technique disclosed in Japanese Laid-openPatent Application No. 2008-028541 because this technique includessuspending line-based wavelet transformation until a necessary number oflines are buffered.

Under the circumstances, there is a need for a technique for reducing atransmission delay caused by image data compression.

SUMMARY OF THE INVENTION

It is an object of the present invention to at least partially solve theproblems in the conventional technology.

An image coding device encodes image data in units of encoding linenumber that is number of lines necessary for implementing encoding. Theimage coding device includes: an image-data transforming unit thattransforms image data of lines, number of which is smaller than theencoding-line number, into to-be-encoded image data of the encoding-linenumber of lines; and an encoding unit that encodes the to-be-encodedimage data.

An image transmitting/receiving system transmits and receives image dataencoded in units of encoding line number that is number being number oflines necessary for implementing encoding. The imagetransmitting/receiving system includes: an image-data transforming unitThat transforms image data of lines, number of which is smaller than theencoding-line number, into to-be-encoded image data of the encoding-linenumber of lines; an encoding unit that encodes the to-be-encoded imagedata; an image-data transmitting unit that transmits encoded data thatis the encoded image data; a decoding unit that decodes the transmittedencoded data; and an image-data reconstructing unit that reconstructsdecoded image data into not-yet-transformed image data.

An image coding method encodes image data in units of encoding linenumber that is number of lines necessary for implementing encoding. Theimage coding method includes: transforming image data of lines, numberof which is smaller than the encoding-line number, into to-be-encodedimage data of the encoding-line number of lines; and encoding theto-be-encoded image data.

The above and other objects, features, advantages and technical andindustrial significance of this invention will be better understood byreading the following detailed description of presently preferredembodiments of the invention, when considered in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example utilization form of an imagetransmitting/receiving system according to an embodiment of the presentinvention;

FIG. 2 is a diagram illustrating a hardware configuration of a networkcamera according to the embodiment;

FIG. 3 is a block diagram illustrating a functional configuration of aconventional network camera or comparison with the network cameraaccording to the embodiment;

FIG. 4 is a block diagram illustrating a functional configuration of thenetwork camera according to the embodiment;

FIG. 5 is a block diagram illustrating configuration of an imagereceiving apparatus according to the embodiment;

FIGS. 6A and 6B are diagrams illustrating how image data is encoded inthe conventional network camera for comparison with an image-datarearranging process according to the embodiment;

FIGS. 7A to 7C are diagrams illustrating how image data is encoded inthe network camera according to the embodiment;

FIG. 8 is a diagram illustrating, in time sequence, an encoding processaccording to the embodiment;

FIGS. 9A and 9B are diagrams illustrating how image data of two lines isencoded in the network camera according to the embodiment;

FIGS. 10A and 10B are diagrams illustrating how image data of four linesis encoded in the network camera according to the embodiment;

FIG. 11 is a graph of compression ratio of to-be-encoded image dataversus data sizes with different numbers of lines used for encoding inthe network camera according to the embodiment;

FIG. 12 is a diagram illustrating a functional configuration of thenetwork camera according to a second embodiment of the presentinvention;

FIG. 13 is a diagram illustrating control information according to thesecond embodiment;

FIG. 14 is a block diagram illustrating a structure of a packetaccording to the second embodiment;

FIG. 15 is a block diagram illustrating a functional configuration ofthe image receiving apparatus according to the second embodiment;

FIG. 16 is a diagram illustrating rearrangement information according tothe second embodiment;

FIG. 17 is a flowchart illustrating operations of elements of thenetwork camera according to the embodiment; and

FIG. 18 is a flowchart illustrating operations of the elements of thenetwork camera according to the embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

A first embodiment of the present invention is described below indetail. In the description given below, the present embodiment isimplemented as an image transmitting/receiving system including an imageprocessing apparatus and an image receiving apparatus. The imageprocessing apparatus that performs image data processing such as imagedata encoding and image data transmission is embodied as a networkcamera. The image receiving apparatus receives image data transmittedfrom the image processing apparatus via a network.

FIG. 1 is a diagram illustrating an example utilization form of theimage transmitting/receiving system. As illustrated in FIG. 1, the imagetransmitting/receiving system according to the present embodimentincludes a network camera 1 and an image receiving apparatus 2 connectedto each other via a network such as the Internet or a LAN (local areanetwork).

The network camera 1 which may be a security camera or a vehicle-mountedcamera, for example, generates data representing a moving image or astill image (hereinafter, collectively referred to as “image”) bycapturing an image of an image capture subject and transmits the data tothe image receiving apparatus 2. The image receiving apparatus 2receives the image data transmitted from the network camera 1 anddisplays the received image data and/or performs image recognition, forexample.

Hardware of the network camera 1 included in the imagetransmitting/receiving system according the present embodiment isdescribed below. FIG. 2 is a block diagram illustrating a hardwareconfiguration of the network camera 1 according to the presentembodiment.

As illustrated in FIG. 2, the network camera 1 according to the presentembodiment includes a CPU (central processing unit) 10, a RAM (randomaccess memory) 20, a ROM (read only memory) 30, a dedicated engine 40,and an I/F 50 connected to each other via a bus 80. An operating unit 60and a network 70 are connected to the I/F 50.

The CPU 10 is a processing unit that controls overall operations of thenetwork camera 1. The RAM 20 is a volatile storage medium to and fromwhich information can be written and read out and used as a working areain information processing performed by the CPU 10. The ROM 30 is aread-only non-volatile storage medium where program instructions such asvarious types of control programs and firmware are stored.

The dedicated engine 40 is hardware for implementing a dedicatedfunction of the network camera 1. The dedicated engine 40 may be, forexample, a processing device performing processing for rearrangingpixels of image data acquired line by line. The processing device may beimplemented in an ASIC (application specific integrated circuit), forexample.

The I/F 50 provides and controls connection between the bus 80 andvarious types of hardware, a network, and the like. The operating unit60 is user interface that allows a user to enter information to thenetwork camera 1. The operating unit 60 may be, for example, varioushard buttons and a touch panel. The network 70 is an interface thatallows the network camera 1 to carry out network communication withother equipment.

In such hardware configuration, the CPU 10 performs processing inaccordance with a program stored in the ROM 30 or a program loaded ontothe RAM 20, thereby configuring software control units. Functionalblocks that implement functions of the network camera 1 included in theimage transmitting/receiving system according to the present embodimentare implemented in combination of the software control units configuredas described above and hardware.

The image receiving apparatus 2 has a hardware configuration similar tothat of an information processing apparatus such as a typical PC(personal computer) or a server. More specifically, the image receivingapparatus 2 includes, in addition to the hardware configurationillustrated in FIG. 2, an LCD (liquid crystal display), which isconnected to the I/F 50, and an HDD (hard disk drive).

A functional configuration of a conventional network camera is describedbelow for comparison purpose prior to describing a functionalconfiguration of the network camera 1 according to the presentembodiment. FIG. 3 is a block diagram illustrating a functionalconfiguration of functions which are related to an image-data encodingprocess and an image-data transmission process and which are provided bythe conventional network camera. In the present embodiment, it isassumed that “encoding process” is encoding involving compression, and“encoding (process)” is equivalent to “compression (process)”.

As illustrated in FIG. 3, the conventional network camera includes acamera 101, an image-data acquisition unit 102, line buffers 103-1 to103-8, an encoding unit 104, a packet generation unit 105, and a networkI/F 106.

The camera 101 is an image capturing device that captures an image usingan imaging capturing element such as a CCD (charge coupled device) or aCMOS (complementary metal oxide semiconductor) and outputs the image asdigital data.

The image-data acquisition unit 102 acquires an image generated by thecamera 101 capturing an image of an image capture subject as digitaldata and outputs the acquired image data to the line buffers 103-1 to103-8 (hereinafter, “the first to eighth line buffers”) line by line.

Each of the first to eighth line buffers is a storage medium where theimage data of one line (hereinafter, sometimes referred to as “linedata”) output from the image-data acquisition unit 102 is to be stored.The first to eighth line buffers, the number of which is equal to thenumber of the lines necessary for the encoding unit 101, which will bedescribed later, to encode image data, are provided. The encoding unit104 encodes image data in units of the encoding line number that is thenumber of lines necessary for implementing encoding (e.g., 8 lines). Forthis reason, the network camera 1 includes the eight (the first toeighth) line buffers.

The encoding unit 104 acquires the line data stored in the first toeighth line buffers and encodes the image data in units of 8 lines usinga JPEG scheme. The encoding unit 104 outputs the encoded image data tothe packet generation unit 105.

More specifically, the encoding unit 104 so its the acquired image dataof 8 lines into blocks of, for example, 8 lines by 8 pixels andtransforms the image data from the spatial domain to the frequencydomain by DCT (discrete cosine transform) block by block. The encodingunit 104 quantizes the transformed data and encodes the quantized databy entropy coding.

The packet generation unit 105 packetizes the encoded image data(hereinafter, “encoded data”) output from the encoding unit 104 toconvert the encoded data to a data format that can be transmitted via anetwork. The packet generation unit 105 also functions as an image-datatransmitting unit that transmits the packetized data (hereinafter,“packet data”) to other equipment such as the image receiving apparatus2 via the network I/F 106.

The network I/F 106 is an interface that allows the network camera 1 tocommunicate with other equipment such as the image receiving apparatus 2over the network. An interface such as an Ethernet (registeredtrademark) interface, a Bluetooth (registered trademark), or a Wi-Fi(wireless fidelity) interface may be used as the network I/F 106.

More specifically, the image-data acquisition unit 102 outputs line datafor the first to eighth lines to the eight (the first to eighth) linebuffers, respectively. When the line data has been stored in the eight(the first to eighth) line buffers, the encoding unit 104 encodes theline data in the line buffers. When the line data in the first to eighthline buffers has been encoded, the image-data acquisition unit 102 thenoutputs line data for the ninth to sixteenth lines to the eight (thefirst to eighth) line buffers, respectively. This procedure isrepeatedly performed until all the line data of the image data isencoded.

As described above, the conventional network camera temporarily stores anecessary number of lines, necessary for encoding, of image data line byline. Therefore, the conventional network camera is required to includeas many line buffers as the lines necessary for encoding, whichincreases circuit size. Furthermore, because an encoding process cannotbe performed until line data is stored in all the line buffers, a delayoccurs in transmission of encoded data.

A feature of the present embodiment lies in an encoding process thatreduces such a delay in transmission of encoded data. As a configurationaccording to the present embodiment, a functional configuration offunctions which are related to the image-data encoding process and animage-data transmission process and which are provided by the networkcamera 1 according to the present embodiment is described below. FIG. 4is a block diagram illustrating the functional configuration of thefunctions involved in the image-data encoding process and the image-datatransmission process out of the functions of the network camera 1according to the present embodiment.

As illustrated in FIG. 4, the network camera according to the presentembodiment differs from the configuration illustrated in FIG. 3 in thatthe eight (the first to eighth) line buffers are replaced with a singleline buffer 103 and that a rearranging unit 107 added.

The camera 101 is similar to the camera 101 described above withreference to FIG. 3. As described above with reference to FIG. 3, theimage-data acquisition unit 102 acquires digital data representing animage generated by the camera 101 capturing an image and outputs theacquired image data of one line to the line buffer 103.

The rearranging unit 107 rearranges the image data of one line stored inthe line buffer 103 into image data (hereinafter, “to-be-encoded imagedata”) made up of the encoding-line number of lines (e.g., 8 lines),which is the number of lines necessary for implementing encoding, andoutputs the rearranged image data to the encoding unit 104. A feature ofthe present embodiment lies in this rearranging process performed by therearranging unit 107. The rearranging process will be described indetail later.

The encoding unit 104 encodes the to-be-encoded image data output fromthe rearranging unit 107. Accordingly, the encoding unit 104 accordingto the present embodiment encodes the image data line by line. A featureof the present embodiment lies in the function as an image coding deviceincluding the rearranging unit 107 and the encoding unit 104 describedabove.

The packet generation unit 105 and the network I/F 106 operate in amanner similar to that described above with reference to FIG. 3.

More specifically, the image-data acquisition unit 102 outputs line datafor the first line to the line buffer 103 first. The line data stored inthe line buffer 103 is rearranged by the rearranging unit 107 intoto-be-encoded image data, which is then encoded by the encoding unit104. When the line data for the first line has been encoded, theimage-data acquisition unit 102 then outputs line data for the secondline to the line buffer 103. This procedure is repeatedly performeduntil all the line data of the image data is encoded.

A functional configuration of the image receiving apparatus 2 accordingto the present embodiment is described below. FIG. 5 is a block diagramillustrating a functional configuration of a function which is relatedto a process of decoding the encoded data transmitted from the networkcamera 1 into image data out of the functions of the image receivingapparatus 2 according to the present embodiment. As illustrated in FTC.5, the image receiving apparatus 2 according to the present embodimentincludes a network I/F 201, a packet analysis unit 202, a decoding unit203, and a rearranging unit 204.

The network I/F 201 is an interface that allows the image receivingapparatus 2 to communicate with other equipment such as the networkcamera 1 over the network. An interface such as an Ethernet (registeredtrademark) interface, a Bluetooth (registered trademark), or a Wi-Fiinterface may be used as the network I/F 201.

The packet analysis unit 202 acquires the packet data transmitted fromthe network camera 1 via the network I/F 201, obtains encoded data fromthe packet data, and outputs the encoded data to the decoding unit 203.

The decoding unit 203 decodes the encoded data output from the packetanalysis unit 202. The encoded data is thus reconstructed by thedecoding unit 203 into the image data of 8 lines rearranged by therearranging unit 107. The decoding unit 203 outputs the reconstructedimage data to the rearranging unit 204.

The rearranging unit 204 rearranges the image data of 8 lines outputfrom the decoding unit 203 into the image data of one line. Morespecifically, the rearranging unit 204 reconstructs the image data of 8lines into the image data of one line by performing, in reverse, theprocedure through which the rearranging unit 107 has rearranged theimage data. That is, the rearranging unit 204 functions as an image-datareconstructing unit that reconstructs the image data decoded by thedecoding unit 203 into the image data that is not rearranged yet by therearranging unit 107. The image data of one line reconstructed by therearranging unit 204 is eventually used in displaying an image on adisplay unit, such as the LCD 60, of the image receiving apparatus 2,for example.

How the rearranging unit 107 according to the present embodimentrearranges image data is described below. FIGS. 6A and 6B are diagramsillustrating, for comparison with an image-data rearranging processaccording to the present embodiment, how image data is encoded in theconventional network camera illustrated in FIG. 3. FIGS. 7A to 7C arediagrams illustrating how image data is encoded in the network camera 1according to the present embodiment.

In the conventional network camera illustrated in FIG. 3, such imagedata of 8 lines as that illustrated in FIG. 6A composed of pieces ofimage data each being of one line and stored in the eight (the first toeighth) respective line buffers is used as image data to be encoded bythe encoding unit 104. The encoding unit 104 divides the image data of 8lines into blocks of 8 lines by 8 pixels as illustrated in FIG. 6A andencodes the image data block by block, thereby outputting such encodeddata as that illustrated in FIG. 6B.

By contrast, in the network camera 1 according to the present embodimentillustrated in FIG. 4, image data one line is stored in the single linebuffer 103 as illustrated in FIG. 7A. In the image data illustrated inFIG. 7A, pixels are filled with hatching patterns changing every 8pixels. These hatching patterns are used for convenience of thedescription of the rearranging process given below, and not intended tomean that pixels with the same hatching pattern are of the same pixelvalue or the like.

For example, the rearranging unit 107 divides the image data of one lineillustrated in FIG. 7A into subblocks of 1 line by 8 pixels andrearranges the subblocks into to-be-encoded image data. The pixels ofthe rearranged image data illustrated in FIG. 7B are filled with thesame hatching patterns as those of the image data illustrated in FIG. 7Achanging every 8 pixels.

More specifically, for instance, the first to eighth pixels, filled withlines rising to the right, of the image data illustrated in FIG. 7A arearranged as the first 8 pixels on the first line of the image dataillustrated FIG. 7B. For instance, the ninth to sixteenth pixels, filledwith least dense dots, of the image data illustrated in FIG. 7A arearranged as the first 8 pixels on the second line of the image dataillustrated in FIG. 7B.

When the rearranging unit 107 has arranged the divided image data of 8pixels to the eighth line in this manner, the rearranging unit 107 thenarranges the image data of the next 8 pixels at a position of the ninthto sixteenth pixels on the first line. By performing the sequencedescribed above, the rearranging unit 107 rearranges the image data ofone line illustrated in FIG. 7A into the image data of 8 linesillustrated in FIG. 7B in real-time. The image data of 8 linesillustrated in FIG. 7B is used as the to-be-encoded image data to beencoded by the encoding unit 104.

In short, the rearranging unit 107 functions as an image-datatransforming unit that divides the image data stored in the line buffer103 into subblocks each made up of a predetermined number of pixels andtransforming the subblocks each made up of the predetermined number ofpixels into to-be-encoded image data.

The encoding unit 104 divides the image data of 8 lines into blocks of 8lines by 8 pixels as illustrated in FIG. 7B and encodes the image datablock by block, thereby outputting such encoded data as that illustratedin FIG. 7G.

FIG. 8 is a diagram illustrating, in time sequence, the conventionalencoding process and the encoding process according to the presentembodiment. In the conventional encoding process, as illustrated at (a)FIG. 8, the encoding unit 104 encodes image data of 8 lines only afterthe image data of 8 lines has been buffered, and transmits thethus-encoded data.

By contrast, in the encoding process according to the presentembodiment, as illustrated at (b) in FIG. 8, each time image data of oneline is acquired, the rearranging unit 107 rearranges the image data.The encoding unit 101 encodes the rearranged image data and transmitsthe thus-encoded data. Furthermore, in parallel with transmission of theencoded data, image data of the next one line is acquired and rearrangedby the rearranging unit 107.

Accordingly, first transmission of encoded data in the encoding processillustrated at (b) in FIG. 8 is earlier than that in the encodingprocess illustrated at (a) in FIG. 8. Furthermore, in the encodingprocess illustrated at (b) in FIG. 8, image data is encoded andtransmitted each time image data of one line is acquired. Accordingly,the encoding process illustrated at (b) in FIG. 8 completes transmissionof encoded data obtained by encoding the image data of 8 lines earlierthan the encoding process illustrated at (a) in FIG. 8. In short, adelay in transmission of encoded data is reduced as illustrated in FIG.8, for example.

FIG. 17 is a flowchart illustrating operations of elements of thenetwork camera 1 illustrated in FIG. 4. The operations in the flowchartillustrated in FIG. 17 are an example of operations for implementing animage coding method. As illustrated in FIG. 17, the image-dataacquisition unit 102 acquires image data of one line (S1701). Uponacquiring the image data of one line, the image-data acquisition unit102 outputs the image data of one line to the line buffer 103 (S1702).

The rearranging unit 107 transforms the image data of one line stored inthe line buffer 103 into to-be-encoded image data of the encoding-linenumber of lines (e.g., 8 lines) (S1703). When the image data of one linehas been transformed by the rearranging unit 107 into the to-be-encodedimage data, the encoding unit 104 encodes the to-be-encoded image data(S1704).

When the to-be-encoded image data has been encoded into encoded data bythe encoding unit 104, the packet generation unit 105 packetizes theencoded data into packet data and transmits the packet data via thenetwork I/F 106 (S1705). The procedure illustrated in FIG. 17 isrepeatedly performed until all the line data is processed.

As described above, the network camera 1 according to the presentembodiment transforms, each time image data of one line is stored in theline buffer 103, the image data of one line into to-be-encoded imagedata of 8 lines, encodes the to-be-encoded image data into encoded data,and transmits the encoded data to the image receiving apparatus 2. Thiscan reduce a transmission delay caused by image data compression asdescribed above with reference to FIG. 8.

In the present embodiment, the example where each time image data of oneline is stored in the line buffer 103, the image data is transformedinto to-be-encoded image data and then encoded is described. However,the number of lines to be transformed into to-be-encoded image data isnot limited to one, but may be two or four, for example, as long as itis smaller than the encoding-line number.

FIGS. 9A and 9B are diagrams illustrating how image data of two linesare encoded. To implement encoding of image data of two lines, thenetwork camera 1 includes two line buffers (the first and second linebuffers). Such image data of two lines as that illustrated in FIG. 9Acomposed of pieces of the image data each being of one line and storedin the respective first and second line buffers is used as image data tobe encoded by the encoding unit 104. In the image data illustrated inFIG. 9A, pixels are filled with hatching patterns changing every 8pixels. These hatching patterns are used for convenience of thedescription of the rearranging process given below, and not intended tomean that pixels with the same hatching pattern are of the same pixelvalue or the like.

For example, the rearranging unit 107 divides the image data of twolines illustrated in FIG. 9A into subblocks of 2 lines by 8 pixels andrearranges the subblocks into to-be-encoded image data as illustrated inFIG. 9B. The pixels of the rearranged image data illustrated in FIG. 9Bare filled with the same hatching patterns as those of the image dataillustrated in FIG. 9A changing every 8 pixels.

More specifically, for instance, the first to eighth pixels on each ofthe first and second lines, filled with lines rising to the right, ofthe image data illustrated FIG. 9A are arranged as the first 8 pixels onthe first and second lines of the to-be-encoded image data illustratedin FIG. 9B.

For instance, the ninth to sixteenth pixels on each of the first andsecond lines, filled with least dense dots, of the image dataillustrated in FIG. 9A are arranged as the first 8 pixels on the thirdand fourth lines of the to-be-encoded image data illustrated in FIG. 9B.

When the rearranging unit 107 has performs arrangement up to the eighthline of the to-be-encoded image data is units of the divided image dataof the subblock of 2 lines by 8 pixels in this manner, the rearrangingunit 107 then arranges the image data of the next subblock of 2 lines by8 pixels at a position of the ninth to sixteenth pixels on the first andsecond lines of the to-be-encoded image data. By performing the sequencedescribed above, the rearranging unit 107 rearranges the image data oftwo lines illustrated in FIG. 9A into the to-be-encoded image data of 8lines illustrated in FIG. 9B.

FIGS. 10A and 10B are diagrams illustrating how image data of four linesare encoded. To implement encoding of image data of four lines, thenetwork camera 1 includes four line buffers (the first to fourth linebuffers). Such image data of four lines as that illustrated in FIG. 10Acomposed of pieces of the image data each being of one line and storedin the respective first to fourth line buffers is used as image data tobe encoded by the encoding unit 104. In the image data illustrated inFIG. 10A, pixels are filled with hatching patterns changing every 8pixels. These hatching patterns are used for convenience of thedescription of the rearranging process given below, and not intended tomean that pixels with the same hatching pattern are of the same pixelvalue or the like.

For example, the rearranging unit 107 divides the image data of fourlines illustrated in FIG. 10A into subblocks of 4 lines by 8 pixels andrearranges the subblocks into to-be-encoded image data as illustrated inFIG. 10B. The pixels of the rearranged image data illustrated in FIG.10B are filled with the same hatching patterns as those of the imagedata illustrated in FIG. 10A changing every 8 pixels.

More specifically, for example, the first to eighth pixels on each ofthe first to fourth lines, filled with lines rising to the right, of theimage data illustrated in FIG. 10A are arranged as the first 8 pixels onthe first to fourth lines of the to-be-encoded image data illustrated inFIG. 10.

For example, the ninth to sixteenth pixels on each of the first tofourth lines, filled with least dense dots, of the image dataillustrated in FIG. 10A are arranged as the first 8 pixels on the fifthto eighth lines of the to-be-encoded image data illustrated in FIG. 10B.

When the rearranging unit 107 has performed arrangement up to the eighthline of the to-be-encoded image data in units of the divided image dataof the subblock of 4 lines by 8 pixels in this manner, the rearrangingunit 107 then arranges the image data of the next subblock of 4 lines by8 pixels at a position of the ninth to sixteenth pixels on the first tofourth lines of the to-be-encoded image data. By performing the sequencedescribed above, the rearranging unit 107 rearranges the image data offour lines illustrated in FIG. 10A into the to-be-encoded image data of8 lines illustrated in FIG. 10B.

FIG. 11 is a graph of compression ratio of to-be-encoded image dataversus data size with different numbers of lines used for encoding inthe network camera 1 according to the present embodiment. As illustratedin FIG. 11, when comparison is made at the same compression ratio, thedata size increases in the following order: image data of one line, twolines, three lines, and not rearranged (i.e., 8 lines).

This can be ascribed to characteristics of JPEG compression. JPEGcompression reduces the amount of data by removing high-frequencycomponents, to which human vision is less sensitive, from an image.Natural images such as images obtained by capturing the images ofoutdoor scenery have a feature that the images have small high-frequencycomponents and, accordingly, compression ratios of these imagescompressed by JPEG are generally high. However, when line data isrearranged as in the present embodiment, because to-be-encoded imagedata has a large number of spatially-discontinuous regions (i.e.,high-frequency components) where color changes sharply, compressionratio of JPEG-compressed images decreases.

Accordingly, as illustrated in FIG. 11, as the number of lines(hereinafter, “the number of rearrangement lines”) of image data to berearranged by the rearranging unit 107 increases, the discontinuousregions decrease, and the compression ratio is improved. Meanwhile, asdescribed earlier, when the number of rearrangement lines is two, twoline buffers are to be used; when the number of rearrangement lines isfour, four line buffers are to be used. In short, as the number ofrearrangement lines increases, the number of the line buffers increases,which increases circuit size. Furthermore, as the number ofrearrangement lines increases, the time it takes until the image data ofrearrangement lines is buffered is lengthened. As a result, a delay intransmission of encoded data increases.

FIG. 18 is a flowchart illustrating operations of elements of thenetwork camera 1 illustrated in FIG. 4. The operations in the flowchartillustrated in FIG. 18 are an example of operations for implementing animage coding method. In the flowchart illustrated in FIG. 18, image datais encoded in units of a number of lines, the number being two or fourin the above-described examples and being smaller than the encoding-linenumber.

As illustrated in FIG. 18, the image-data acquisition unit 102 acquiresimage data of one line (S1801). Upon acquiring the image data of oneline, the image-data acquisition unit 102 outputs the image data of oneline to a line buffer (S1802).

The rearranging unit 107 determines whether or not image data of apredetermined number of lines necessary or encoding have been stored inthe line buffer(s) (S1803). For instance, when the necessary number oflines is two, the rearranging unit 107 determines whether or not linedata has been stored in each of the first and second line buffers. Ifthe image data of the predetermined number of lines has not been storedin the corresponding line buffer(s) yet (NO at S1803), the rearrangingunit 107 is held on standby. The image-data acquisition unit 102acquires image data of the next one lice (S1801) and outputs the imagedata of one line to another line buffer (S1802).

If the image data of the predetermined number of lines has been storedin the corresponding line buffer(s) (YES at S1803), the rearranging unit107 transforms the line data stored in the line buffer (s) intoto-be-encoded image data of the encoding-line number of lines (e.g., 8lines) (S1804).

When the image data of the predetermined number of lines has beentransformed by the rearranging unit 107 into the to-be-encoded imagedata, the encoding unit 104 encodes the to-be-encoded image data(S1805). When the to-be-encoded image data has been encoded into encodeddata by the encoding unit 104, the packet generation unit 105 packetizesthe encoded data into packet data and transmits the packet data via thenetwork I/F 106 (S1806). The procedure illustrated in FIG. 18 isrepeatedly performed until all the line data is processed.

Second Embodiment

A second embodiment of the invention is described below. As describedabove in the first embodiment, it is enough that the number ofrearrangement lines is smaller than the encoding-line number.Furthermore, as described above with reference to FIG. 11, as the numberof rearrangement lines increases, the compression ratio is improved; inother words, the size of data compressed the same compression ratiodecreases.

The number of rearrangement lines can be determined depending on anetwork communication status of the image transmitting/receiving systemby utilizing these features. More specifically, when, for instance,communications traffic on the network is light (or, in other words, whenthe amount of data that can be transmitted is large), the number ofrearrangement lines is determined to one. When the number ofrearrangement lines is one, although the compression ratio of image datais lower (or, in other words, the size of data compressed at the samecompression ratio is larger) than that when the number of rearrangementlines is two or larger, a delay in transmission of encoded datadecreases.

When, for instance, communications traffic on the network is busy (or,in other words, when the amount of data that can be transmitted issmall), the number of rearrangement lines is determined to four. Whenthe number of rearrangement lines is four, although a delay intransmission of encoded data is larger than that when the number ofrearrangement lines is smaller than four, the compression ratio of imagedata is improved (or, in other words, the size of data compressed at thesame compression ratio decreases).

In the second embodiment, the number of rearrangement lines isdetermined depending on the amount of data that can be transmitted via anetwork. FIG. 12 is a block diagram illustrating a functionalconfiguration of the network camera 1 according to the secondembodiment. As illustrated in FIG. 12, the network camera 1 according tothe second embodiment differs from the configuration illustrated in FIG.4 in additionally including a control signal analysis unit 108 and acontrol-information storage unit 109. Description of elements thatperform similar operations to those illustrated in FIG. 4 is omitted.

In the second embodiment, the number of rearrangement lines ischangeable to any one of one, two, and four. Accordingly, the networkcamera 1 includes the four line buffers 103 and uses one or more of theline buffers 103 the number of which corresponds to the rearrangementlines.

The control signal analysis unit 108 receives a control signaltransmitted from a network monitoring apparatus 3 via the network I/F106 and determines the number of rearrangement lines. After determiningthe number of rearrangement lines, the control signal analysis unit 108outputs the thus-determined number of rearrangement lines to theimage-data acquisition unit 102, the rearranging unit 107, and thepacket generation unit 105.

More specifically, for instance, the network monitoring apparatus 3transmits information indicating a level of volume of traffic(hereinafter, “traffic level”), which is obtained by monitoring networktraffic or the like on the image transmitting/receiving system, as acontrol signal to the network camera 1. For instance, volumes of networktraffic may be classified in advance into the following three trafficlevels: “0”, “1”, and “2”, from smallest to largest traffic volume.

Upon receiving a control signal indicating such a traffic level as thatdescribed above, the control signal analysis unit 108 retrieves thenumber of rearrangement lines associated with the traffic level from thecontrol-information storage unit 109. That is, the control signalanalysis unit 108 functions as a line-number determining unit thatdetermines the number of rearrangement lines depending on communicationstatus information indicating a network communication status.

FIG. 13 is a diagram illustrating an example of control informationstored in the control-information storage unit 109. As illustrated inFIG. 13, the control information is configured as a table where eachtraffic level is associated with the number of rearrangement lines. Forinstance, when the traffic level is “0”, the system is in a conditionwhere the volume of network traffic is small and even if a large volumeof data is transmitted, a delay is less likely to occur. Put anotherway, in this condition, the amount of data that can be transmitted islarge. Accordingly, the number of lines “1” with which the compressionratio is low but transmission delay is small is associated with thetraffic level “0”. For instance, when the traffic level is “2”, thesystem is a condition where the volume of network traffic is large andit is desirable to minimize the amount of data to be transmitted.Accordingly, the line number “4” with which transmission delay is largebut the compression ratio is high is associated with the traffic level“2”.

The image-data acquisition unit 102 outputs image data of lines, thenumber of which is the number of rearrangement lines output from thecontrol signal analysis unit 108, to the line buffers 103, the number ofwhich is the number of rearrangement lines, line data by line data. Whenthe image data of the lines, the number of which is the number ofrearrangement lines output from the control signal analysis unit 108,has been stored in the line buffers 103, the rearranging unit 107rearranges the image data of the lines, the number of which is thenumber of rearrangement lines, into to-be-encoded image data and outputsthe to-be-encoded image data to the encoding unit 104.

The packet generation unit 105 adds a control signal output from thecontrol signal analysis unit 108 to the encoded data output from theencoding unit 104 to perform packetizing. FIG. 14 is a diagramillustrating a configuration of a structure of a packet generated by thepacket generation unit 105. As illustrated in FIG. 14, the packetgenerated by the packet generation unit 105 according to the secondembodiment contains a rearrangement ID by which the number ofrearrangement lines can be identified. For instance, when the number ofrearrangement lines is “1”, the rearrangement ID may be set to “0”; whenthe number of rearrangement lines is “2”, the rearrangement ID may beset to “1”; when the number of rearrangement lines is “3”, therearrangement ID may be set to “2”.

A functional configuration of the image receiving apparatus 2 accordingto the second embodiment is described below. FIG. 15 is a diagramillustrating an example of the functional configuration of the imagereceiving apparatus 2 according to the second embodiment. As illustratedin FIG. 15, the image receiving apparatus 2 according to the secondembodiment differs from the configuration illustrated in FIG. 5 inadditionally including a rearrangement-information storage unit 205.Description of elements that perform similar operations to thoseillustrated in FIG. 5 is omitted.

FIG. 16 is a diagram illustrating an example of rearrangementinformation stored in the rearrangement-information storage unit 205. Asillustrated in FIG. 16, the control information is configured as a tablewhere each rearrangement ID, which is contained in the packetillustrated in FIG. 14, is associated with the number of rearrangementlines.

The packet analysis unit 202 acquires packet data transmitted from thenetwork camera 1 via the network I/F 201 and acquires encoded data and arearrangement ID from the received packet. The packet analysis unit 202determines the number of rearrangement lines by retrieving the numberassociated with the acquired rearrangement ID from therearrangement-information storage unit 205. For instance, when theacquired rearrangement ID is “0”, the packet analysis unit 202 candetermine that the acquired encoded data is image data of one line.

Furthermore, the packet analysis unit 202 outputs the acquired encodeddata to the decoding unit 203, and outputs the determined number ofrearrangement lines to the rearranging unit 204. The rearranging unit204 rearranges the image data of 8 lines output from the decoding unit203 into image data of lines, the number of which is the number ofrearrangement lines output from the packet analysis unit 202.

As described above, in the second embodiment, the number ofrearrangement lines is determined depending on a network communicationstatus of the image transmitting/receiving system. Thus, the secondembodiment makes it possible to determine whether to prioritize a highcompression ratio over a small transmission delay or vice versa, therebytransmitting data efficiently depending on a network status whilereducing transmission delay.

The second embodiment has been described by way of the example in whichthe control signal is transmitted from the network monitoring apparatus3, which is an apparatus independent of the network camera 1.Alternatively, configuration in which the network camera 1 itselfmonitors the network traffic status and determines the number ofrearrangement lines depending on the status may be employed.

In the first and second embodiments, the network camera 1 is used as anexample of the image processing apparatus. However, the image processingapparatus is not limited to the network camera 1 but can be any imageprocessing apparatus so long as having a configuration for encodingimage data and transmitting the encoded image data to other equipment.

An embodiment is a computer program product comprising a non-transitorycomputer-readable medium containing an information processing program,the program causing a computer of an image coding device that encodesimage data in units of encoding line number that is number of linesnecessary for implementing encoding, to perform: transforming image dataof lines, number of which is smaller than the encoding-line number, intoto-be-encoded image data of the encoding-line number of lines; andencoding the to-be-encoded image data.

According to an embodiment, a transmission delay caused by image datacompression can be reduced.

Although the invention has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. An image coding device that encodes image data inunits of an encoding line number that is a number of lines necessary forimplementing encoding, the image coding device comprising: processingcircuitry configured to transform image data of lines, a number of whichis smaller than the encoding-line number, into to-be-encoded image dataof the encoding-line number of lines, and encode the to-be-encoded imagedata.
 2. The image coding device according to claim 1, wherein theprocessing circuitry determines the number of lines of the image data tobe transformed into the to-be-encoded image data depending oncommunication status information indicating a communication status of anetwork over which the encoded image data is to be transmitted, andtransforms the image data of the determined number of lines into theto-be-encoded image data.
 3. The image coding device according to claim2, wherein the communication status information indicates an amount ofdata transmittable over the network, and the processing circuitrydetermines the number of lines such that the larger the amount oftransmittable data, the smaller the number of lines.
 4. The image codingdevice according to claim 1, wherein the processing circuitry divideseach line of the image data to be transformed into subblocks, each ofthe subblocks being made up of a predetermined number of pixels, andrearranges the subblocks into image data of the encoding-line number oflines.
 5. The image coding device according to claim 1, wherein theimage data is generated by capturing, by an image capturing device, animage of an image capture subject, and the processing circuitrytransforms the generated image data in real-time.
 6. An image processingapparatus comprising: the image coding device according to claim 1,wherein the processing circuitry transmits the encoded image data toanother apparatus.
 7. An image transmitting/receiving system thattransmits and receives image data encoded in units of encoding linenumber that is a number of lines necessary for implementing encoding,the image transmitting/receiving system comprising: processing circuitryconfigured to transform image data of lines, a number of which issmaller than the encoding-line number, into to-be-encoded image data ofthe encoding-line number of lines, encode the to-be-encoded image data;transmit encoded data that is the encoded image data, decode thetransmitted encoded data, and reconstruct decoded image data intonot-yet-transformed image data.
 8. An image coding method that encodesimage data in units of encoding line number that is a number of linesnecessary for implementing encoding, the image coding method comprising:transforming, via processing circuitry, image data of lines, a number ofwhich is smaller than the encoding-line number, into to-be-encoded imagedata of the encoding-line number of lines; and encoding, via theprocessing circuitry, the to-be-encoded image data.